This invention also reduces the impact of packet loss on video data quality. For a fixed 
packet drop rate, the video data quality is improved by reducing the number of high 
priority packets being dropped. 

Additionally, the invention intelligently selects packets to deliver so that a higher 
video data quality may be obtained when not all the packets can be transmitted on time. 

By selecting a threshold h appropriately, in most cases, the invention can perform 
within 1-3% of the optimum result determined by the channel conditions and operation is 
significantly improved as the result of utilizing packet selection. 

BRIEF DESCRIPTION OF THE DRAWrNGS 

The exemplary embodiments of this invention will be described in detail, with 
reference to the following figures, wherein: 

Fig. 1 illustrates a video data transmission system according to a preferred 
embodiment of the invention; 

Fig. 2 illustrates a two state Markovian channel model; 

Fig. 3 illustrates a representation of a video data stream, where each column 
represents one frame while each row represents one layer; 

Fig. 4 illustrates an exemplary method of determining whether to transmit a 
packet by estimating the value of success probability as used in the first exemplary 
embodiment; 

Fig. 5 shows exemplary constant bit rate video data used to illustrate the 
application of the exemplary embodiments; 

Fig. 6 shows exemplary variable bit rate video data used to illustrate the 
application of the exemplary embodiments; 

Figs. 7(a) - 7(e) show simulation results with the quality ind e x plotted as a function 

of s yst e m load for diff e rent chann Q l condition transition probability p cG- ^nd for thresholds 
/i ^ 0.1, 0.5, 0.7, 0.9 and 0.99, respectively; 

Fig. 7 % illustrates an exemplary method of determining whether to transmit a 
packet by estimating the value of success probability including operation steps for 
backing up the transmission as used in the second exemplary embodiment; 

Figs. 9(a) 9(e) show simulation results using th e paramotors used in the 

simulation of Fig. 7(a) 7(e) for the method illustrat e d in Fig. 8; 
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FigG 10(a) 10(c) show simulation results using a constant bit rate vid e o data filo 

with the threshold h being s ot at 0.7, 0.9, and 0.99, rospectivoly, for tho method illustrated 
in Fig. 4; and 

Figs 1 1(a) 1 1(e) show simulation results using the param e t e rs used in th e 

s imulation of Figs, 10(a) 10(c) for tho method illustrated in Fig. 8. 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

With a same amount of data, variable bit rate (VBR) encoding can generally 
achieve better video data quality than constant bit rate (CBR) encoding. However, 
efficient transmission of VBR video data has always been among the most challenging 
problems to network designers, largely because of the video data's high bandwidth 
demand, the quality of signal (QoS) requirements and the significant rate of variability. 

In the present invention, video data smoothing is used as an effective way to 
reduce the variability of the bandwidth requirement for transmitting the video data, which 
can potentially simplify other operations such as resource allocation and improve 
network utilization. 

Video data smoothing preloads part of the video data to a smoothing buffer at the 
client before play-out. After play-out has started, the rest of the video data may be 
transmitted in a less bursty fashion without compromising the quality of the video data. 
For given video data, video data smoothing generates the transmission schedule, which 
includes the rates at which the video data will be delivered during play-out, based on 
buffer size, available bandwidth and allowed play-out delay. A valid transmission 
schedule must guarantee that, given the bandwidth it requires, the smoothing buffer will 
not overflow or underflow during the entire play-out of the video data. Depending on the 
user's requirements, the smoothing algorithm may also need to optimize certain 
characteristics of the transmission schedule, such as peak rate, number of rate changes, 
etc. 

When video data needs to be delivered over a wireless network, it becomes even 
more important to perform smoothing. Wireless channels typically have smaller capacity 
than wired links, which limits the gain provided from multiplexing. By reducing the 
burstiness of the bandwidth requirement through smoothing, more streams may be 
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packet 2000 bits or lower depending on the actual transmission rate. Equation (1) and (2) 
were used to estimate the value of success probability SPj. , with R = 28. 

The performance of any method or system for packet selection largely relies upon 
the threshold h used. Therefore, the relationship of the performance and the chosen 
threshold h will now be explained under various network conditions. Specifically, the 

value of Pec was varied from 0.01 to 0.91 in steps of 0. 1 and the case with pcc = 0.99 

also considered. Also, for a fixed pcc> the value of Pbb was adjusted to obtain different 
network loads. 

For each pair ofpcc and p^B. the transmission performance was investigated by 
measuring the output video data's g/ using five different thresholds A = 0.1 (Fig. 7(a)) , 
0.5 (Fig. 7(b)) , 0.7 (Fig. 7(c)) , 0.9 (Fig. 7(d)) and 0.99 (Fig. 7(o)). Figs. 7(a) 7( o ) show 
the roGults with tho g/ plotted as a function of syst e m load for different p cG- ^nd. thr e sholdG 
k Each graph corr e sponds to a fixed threshold /i, and for each cur\^c in a graph, p ^g-^ 
fix e d. 

Comparing the data graphs in Figs. 7(a) 7(o) , the QI, which indicates the 
transmission performance, first increases as the threshold h increases, then as the 
threshold h approaches one, it starts to decrease. This is an expected result because, with 
a small threshold value h, the packet selection system tends to transmit more packets for 
each frame before advancing to the next one. This means by the time the system is ready 
to transmit later frames, it may not have enough time for even their high priority 
components. Therefore, some high priority packets may not be delivered because the 
system spent too much time on the earlier lower priority packets. Therefore, the 
performance of the system may be improved. 

As the threshold value h increases, less low priority packets are transmitted which 
leaves more room for higher priority packets behind them, thus the performance 
improves and the Q/ increases. However, if threshold h continues to increase, the system 
approaches the other extreme state. It becomes more and more likely that only the 
highest priority layer will be sent, even if some less important packets could have been 
delivered without affecting the outcome of the future frames. As a result, the 
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performance degrades with the increasing threshold value /?. From the data In Figs. 7(a) 
7(a), the performance tops at around A = 0.7. 

The exact threshold value h at which packet selection provides optimum 
transmission performance depends upon channel conditions and may change for different 
video data streams. As packet selection has been described so far, it is possible that the 
packet selection system decides to skip the rest of a frame and start sending layer 0 of the 
next frame, but no packet of this new frame has yet arrived at the base-station. While 
waiting for these packets, the bandwidth, which is otherwise wasted, could have been 
used to deliver skipped packets. Simulation results indicate that the problem becomes 
more and more likely to occur as the threshold h increases. Thus, it is also responsible 
for the rapid degradation of performance seen in Figs. 7(a) 7(o) as the threshold h 
approaches one. 

One way of improving the efficiency of bandwidth usage in this situation is to go 
back and transmit packets that were previously skipped. This operation is referred to as 
"backing up" the transmission. Here, it is assumed that packets skipped are kept in the 
base-station buffer until after their play-out time or are replaced by newly arrived 
packets. As soon as the awaited high priority packet arrives, the original packet selection 
steps are resumed and the new packet is delivered. Transmission may be backed up 
whenever similar situations occur. 

If backing up is employed to further improve transmission, the method illustrated 
in Fig. 4 is altered to include additional steps as shown in Figure 7 S. In this second 
exemplary embodiment shown in Fig. 7 %, the method begins in step 800 and control 
proceeds to step 810, In step 810, it is determined whether the frame /r will be played 
before the end of the next mini slot. If frame k will not be played before the end of the 
next mini slot, control proceeds to step 812. Otherwise, control proceeds to step 900 880 . 
In step 900 880, transmission of layer 0 of the next frame that will not be played before 
the end of the next mini slot is performed and control proceeds to step 9+0 885 . 

In step 812, it is determined whether layer 0 of frame A: + 1 or layer i of frame k is 
available for transmission. If so, control proceeds to step 815. Otherwise control 
proceeds to step 850, explained in detail below. 
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Figures 9(a) 9(e) ahovv th o oimulation Simulation results of the second exemplary 
embodiment of the invention, with the backup procedure incorporated, were obtained 
using the same parameters used earlier, and in the simulation of Figo. 7(a) 7(o) . Fig. 9(a) 
illuotratoa tho simulation results for /; = 0.1, Fig. 9(b) corrosponds to ^ = 0.5, Pigr-9(6) 
corrcGpond G to h = 0.7, Fig. 9(d) corr e sponds to h = 0.9 and Fig. 9(o) correopondG to h = 
0.99. 

The backup procedure always chooses, among all the skipped packets, the one 
with highest priority and lowest index to deliver. Figs. 7(a) 7(c) The data show that, 
using a small threshold value h, the performance of the second embodiment is not much 
altered from the performance of the first embodiment. However, for higher thresholds h, 
performance improves significantly. For example, for some combinations ofpcc and p^, 
the QI obtained with h = 0.9 (Fig. 9(d)) exceeds that when h = 0.7 (Fig. 9(c)) . 

Moreover, with such an improvement, the performance also becomes more stable. 
In particular, performance is within 1-3% of the optimum result over a relatively large 
load range, with h = 0.7 (Fig. 9(c)) and h = 0.9 (Fig. 9(d)) . This large range makes it 
easier to select an appropriate threshold value h for different video data streams. 

Comparing h = 0.\ (Fig. 9(a)) data with h = 0.5-0.99 data (Figs. 9(b) 9(o)) , it 
should be clear that use of a very small threshold h results in consistently poor 
performance. For any fixed pcc, the corresponding ettfve data for /? = 0.1 in Fig. 9(a) is 
always the lowest among the data all five graphs . This substantiates a conclusion that 
systems that do not implement packet selection as used in the present invention 
(equivalent to the case of A = 0), such as conventional transmission systems, do not 
perform well. 

The actual amount of improvement from implementing the packet selection 
provided by the exemplary embodiments depends upon the channel conditions and the 
selected threshold value h. Moreover, improvement is also generally higher for higher 
Pcc and higher network loads. 

Additionally, Figs. 7(a) 7(o) the data indicate that , for somo curves, as system 
load increases, the QI sometimes increases with the load, while sometimes decreases with 
it, especially ath = 0.9 (Fig. 7(d)) and h = 0.99 (Fig. 7(c)) . This correlation, or inverse 
correlation, phenomenon is even more apparent in Figur e s 10(a) 10(c) which graph 
another simulation that used a computer generated CBR video data file that included 
4000 equal-sized frames and the size of a frame is 2000 bits. In each frame, 1000 bits 
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Figures 9(a) 9(o) ohow tho oimulation Simulation results of the second exemplary 
embodiment of the invention, with the backup procedure incorporated, were obtained 
using the same parameters used earlier, and in tho simulation of Figs. 7(a) 7(o) . Fig. 9(a) 
ilhi gtratoG the simulation ihmU i, for h = 0.1, Fig. 9(b) corrosponds io h = 0.5, Figr-^ 
corrospondG to h = 0.7, Fig. 9(d) corr o sponds to h = 0.9 and Fig. 9(o) corresponds to h = 
0.99. 

The backup procedure always chooses, among all the skipped packets, the one 
with highest priority and lowest index to deliver. Figs. 7(a) 7(o) The data show that, 
using a small threshold value h, the performance of the second embodiment is not much 
altered from the performance of the first embodiment. However, for higher thresholds h, 
perfonnance improves significantly. For example, for some combinations of pcc ^ndp^s, 
the QI obtained with h = 0.9 (Fig. 9(d)) exceeds that when h = 0.7 (Fig. 9(c)). 

Moreover, with such an improvement, the performance also becomes more stable. 
In particular, performance is within 1-3% of the optimum result over a relatively large 
load range, with h = 0.7 (Fig. 9(o)) and h = 0.9 (Fig. 9(d)) . This large range makes it 
easier to select an appropriate threshold value h for different video data streams. 

Comparing h=0.l ( Fig. 9(a)) data with h = 0.5-0.99 data (Figs. 9(b) 9(o)), it 
should be clear that use of a very small threshold h results in consistently poor 
performance. For any fixed pco, the corresponding etifve data for A = 0. 1 in Fig. 9(a) is 
always the lowest among the data all five graphs . This substantiates a conclusion that 
systems that do not implement packet selection as used in the present invention 
(equivalent to the case of/? = 0), such as conventional transmission systems, do not 
perform well. 

The actual amount of improvement from implementing the packet selection 
provided by the exemplary embodiments depends upon the channel conditions and the 
selected threshold value h. Moreover, improvement is also generally higher for higher 
Pcc and higher network loads. 

Additionally, Figs. 7(a) 7(o) the data indicate that , for some cur\^0G, as system 
load increases, the QI sometimes increases with the load, while sometimes decreases with 
it, especially ath = 0.9 (Fig. 7(d)) and h = 0.99 (Fig. 7(o)) . This correlation, or inverse 
correlation, phenomenon is even more apparent in Figur e s 10(a) 10(c) which graph 
another simulation that used a computer generated CBR video data file that included 
4000 equal-sized frames and the size of a frame is 2000 bits. In each frame, 1000 bits 
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were of the highest priority, and for the rest of 1000 bits, half were of priority 1 and the 
other half were priority 2. The simulation assumed a play-out period of 6 mini-slots. 
One frame was sent from the server, hence one was received at the base-station, every 6 
mini-slots. The size of the wireless packet was 500 bits and that of a regular packet was 
equal to the frame size. The initial buffer occupancy at the client was 6000 which was 
exactly the total of the first three frames. The A simulation of Figs. 10(a) 10(c) was 
performed to investigate how 0/ changes with load by fixing the value of pcc and 
changing pse for different thresholds ^ h [.] 

Figs. 10(a) 10(c) ^ho\y the results of this simulation with the threshold h bemg set 
at 0.7, 0.9, and 0.99, respectfully. It is clearly shown in Figs 10(a) 10(c) The data shows 
that most of the curves drop suddenly once or twice as the load increases from 0.6 to 2. 
To provide more specific results, Table 1 lists the total number of packets transmitted in 
each layer when pcc^^^H for the three thresholds h - 0.7, h - 0.9 and h = 0.99. This 
data indicates that the reason why the curves drop suddenly at some points, when Pbb 
exceeds certain value, is that almost an entire layer is lost. 

The exact value of this critical point for Pbb depends on pcc and threshold h. For 
instance, when threshold h equals 0.7, starting from Pbb = 0.71, almost the entire layer 2 is 
dropped. When h = 0.9, most of the layer 2 packets are dropped starting from psa = 0.41 ; 
and from Pbb = 0.71, all layer 1 packets are also dropped. While for h = 0.99, the two 
drops take place even earlier at Pbb = 0.11 and Pbb = 0.3 1 respectively. Right after each 
drop, the number of packets transmitted in each layer changes very little as Pbb increases. 
However, the load increases with Pbb^ which means the best quality one can achieve fi-om 
the system actually worsens rather than improves as Pbb increase. Therefore, the relative 
performance of the system becomes better, the QI increases and the curve goes up with 
the load. 

An explanation for these sudden drops is now provided. In the exemplary 
embodiments, as explained above, P^^ is calculated to determine which packet to 

deliver at the end of each mini-slot. For a long CBR video data stream, such as that used 
in the simulation associated with Figs. 10(a) 10(e), calculation of: 
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^ _ {layer 0 size = layer \ size)* {number of frames left)*R 
' {wireless packet size)* {number of mini -slots left) 

is performed to decide whether to select a layer 2 packet, and calculation of: 

^ _ {layer 0 size = layer I size)* {number of frames left)* R 
° {wireless packet size) * {number of mini - slots left) 

is performed to decide whether to select a layer 1 packet. For a CBR video data stream, 
the only factor that may vary in the above two formulas is the ratio between the number 
of remaining frames and the number of remaining mini-slots. 

Since only one frame is sent firom the server in every time slot, with a play-out 
delay of just 3 time slots, a packet does not arrive at the base station until a few time slots 
before its play-out time. Therefore, the ratio, X, , does not change much during the 
entire play-out. As a result, the estimated success probability P^^ ^ also does not change 
appreciably during the entire play-out. Therefore, with a fixed threshold h, it is likely 
that the minimum success probability of a layer remains either above or below the 
threshold h during the entire play-out. In the latter case, an entire layer is dropped. Since 
layer 2 packets need to consider one more layer than those of layer 1, their success 
probabihties are more likely to fall below the threshold h. 

Therefore, the cause of these sudden drops is that, again, the system is waiting for 
higher priority packets which have not arrived at base-station, while it could be sending 
some skipped lower priority packets. As described previously, this problem is remedied 
by the second exemplary embodiment shown in Fig. 7 8. 

The simulation results of the second exemplary embodiment, using the same 
parameters as used in the previous simulation of Figs. 10(a) 10(o), aro shown in Figo 
11(a) 11 (o). Although show that although the curves still drop and then increase at about 
the same points as thooo in Fig s . 10(a) 10(c) , the drops are to a much less significant 
degree. The use of the backup procedure results in improvements that are especially 
significant for large threshold values h. When using the backup procedure in the second 
exemplary embodiment of the invention, the performance is predominantly within 5% of 
the optimum result even for h = 0.9 and h = 0.99. 
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The dogrco of similarity between Figs. 1 1(b) and 1 1(c) is also worth noting. This 
similarity occurred bccauso The data shows that the majority of the low priority packets 
have success probabilities that are less than both thresholds h. Therefore, the majority of 
the low priority packets are transmitted through the backup procedure of steps 850-880 in 
the method illustrated in Fig. 8 7. As a result, the exact value of the threshold h is less 
significant than in the first illustrative embodiment (Fig. 4), which does not use a backup 
procedure. 

The 'bumpy' results in Figs. 7(a) 7(o) are duo to a similar reason. Howovor, th e 
The success probability SPj^ , does vary over a relatively large range during play-out, and 
the dropping effect is not as dramatic and more complex because the video data used in 
that simulation includes different frame sizes. By using the second illustrative 
embodiment of the invention, with the incorporated backup procedure, the sudden drops 
in performance almost completely disappear as shown by a comparison between Figs. 
7(a) 7(o) and Figs. 9(a) 9(c) . 

The wireless channels used in the simulations can be fully characterized by the 
value of pcG and ps^. A comparison of the performance of the exemplary embodiments of 
the invention under different system loads is now provided. This comparison explains 
how the perfomiance is affected by pcc and psBy as well as how to select the threshold h to 
obtain better results. 

Using the first exemplary embodiment of the invention, Figs. 7(a) 7(c) show data 
shows that, under the same load, performance is generally better for small pcc. i.e., when 
both good periods and bad periods are short, than for larger pcc. i.e., when the system 
usually remains in the good state for a long time, but once in the bad state, does not return 
to the good state for a significant period of time. However, using the second exemplary 
embodiment, with a backup procedure incorporated, the difference between various 
combinations of pcc and pss becomes almost negligible. This is especially true for h = 0.7 
and h = 0.9 , as shown in Figs. 9(a) 9(c) . In other words, with a threshold h between 0.7 
and 0.9, the transmission is performed close to optimum under most of the channel 
conditions. The only exception is when pcc is very high, e.g., above 0.99. Figs. 9(a) 
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9^ The data show that, at pcc- 0.99, the performing packet selection provides almost 
no performance improvement unless the threshold h is also set to be very high. In actual 
implementation, setting the threshold h between 0.7 and 0.9 will obtain near optimum 
performance in most cases. An exception is when pccis close to 1, in which case the 
threshold h also needs to be set to be very close to 1 . 

Simulation results indicate that by selecting the threshold h appropriately, the first 
exemplary embodiment of the invention achieves within 5% of the optimum result under 
most channel conditions. Moreover, simulation results also indicate that implementing 
the second exemplary embodiment of the invention, which includes a backup procedure 
that uses an otherwise idle network to deUver skipped packets, further improves 
performance to within 1-3% of optimum transmission capability. Additionally, using the 
second exemplary embodiment of the invention provides a more stable system that is 
capable of using a large range of threshold values h under different network conditions. 
Therefore, the second exemplary embodiment of the invention allows more opportunity 
to choose thresholds h for different video data streams. 

While this invention has been described in conjunction with the specific 
embodiments outlines above, it is evident that many alternatives, modifications and 
variations will be apparent to those skilled in the art. Accordingly, the preferred 
embodiments of the invention, as set forth above, are intended to be illustrative, not 
limiting. Various changes may be made without departing from the spirit and scope of 
the invention. 

For example, the present invention may be employed with various coding 
schemes. Some coding schemes use interframe techniques. For instance, in MPEG, a B 
frame may need the I or P frame behind it in order to be decoded. Therefore, efficiently 
transmitting the B frame by itself does not improve video data quality if the 
corresponding / or P frames are missing. Therefore, the order in which compressed 
pictures are found in the video data stream may be different from the order in which they 
are played. 

Initially, this might suggest that the present invention may not be utilized in such 
interframe coding schemes. However, the benefits of the present invention may be used 
in such coding schemes with minor modifications to the system and method. 



